IMAGE PROCESSING APPARATUS, IMAGE DATA PROCESSING 
METHOD, AND RECORDING MEDIUM 



CROSS-REFERENCE TO RELATED APPLICATIONS 
5 The present document incorporates by reference the entire 

contents of Japanese priority document, 2002-358430 filed in Japan on 
December 10, 2002. 

BACKGROUND OF THE INVENTION 

10 1) Field of the Invention 

The present invention relates to an image processing system 
including a transmission-side image processing apparatus that embeds 
information having been deleted from image data in the image data and 
outputs the image data in a standard format, and reception-side image 

15 processing apparatuses as primary and secondary targets that restore 
the image data based on the information extracted from the input 
standard format image data. 

2) Description of the Related Art 

20 In an image processing system including a transmission-side 

image processing apparatus that reads a paper document by a scanner 
and transmits electronic image data and a reception-side image 
processing apparatus that receives and visualizes the transmitted 
image data, it is generally known that the reception-side image 

25 processing apparatus includes a reception-side image processing 



apparatus as a primary target and a reception-side image processing 
apparatus as a secondary target. It is assumed here that the 
reception-side image processing apparatus of the system is a display 
device if used as the primary target (main target), and is a printer if 
5 used as the secondary target (auxiliary target). The transmission-side 
apparatus converts basic specification (resolution, number of colors, or 
sampling frequency) of image data obtained by scanning so as to match 
with equipment such as a monitor used for the primary target, and 
transmits converted data. In this case, the conversion is performed so 

10 as to reduce mainly the value of the specification. 

The value of basic specification of an image required to be 
displayed with appropriate quality in the reception-side apparatus is 
different depending on the equipment. As for resolution, the printer 
generally requires a larger specification value than the monitor. 

15 Assume the case where the reception-side apparatus as the secondary 
target requires a larger specification value than that of the 
reception-side apparatus as the primary target, if the transmission-side 
apparatus reduces the specification value so as to match with the 
apparatus as the primary target, the image quality cannot be ensured in 

20 the apparatus as the secondary target. This is because image data 
cannot be restored to high quality image data with the large 
specification value again from the image data whose specification value 
has been reduced. In other words, the deleted information cannot be 
restored again. 

25 However, even in the above case, there are two image data 



converting methods capable of ensuring image quality in the image 
processing apparatus as the secondary target. 

One of the methods is to convert basic specification so as to 
match with the image processing apparatus as the secondary target 
5 (first conventional method). The other method is to convert basic 
specification into two files of image data that is converted so as to 
match with the image processing apparatus as the primary target and of 
information data that is deleted through the conversion, and restoring 
image data with high image quality from both of the data (second 

10 conventional method). 

However, the first conventional method has a defect such that 
the equipment as the primary target has to convert the basic 
specification of the image data. The second conventional method has 
a defect such that the two files make the operation complicated. 

15 A method similar to the above methods is disclosed in Japanese 

Patent Application Laid Open No. : 2001-127985. According to this 
method disclosed in this publication, a transmission-side image 
processing apparatus embeds histogram of information for an electronic 
watermark in image data, and a reception-side image processing 

20 apparatus extracts the histogram and subjects it to color process based 
on the information to obtain an image with high quality. The object of 
this method is to take the load of generating the histogram off the 
reception side by putting the load only on the transmission side. 
However, in this method, although the file is made to one, the image 

25 quality cannot be ensured in the equipment as the secondary target, 



only because information is embedded in image data in such a manner 
that the information can also be generated from the received image 
data (identical information is doubly included). 



5 SUMMARY OF THE INVENTION 

It is an object of the present invention to solve at least the 
problems in the conventional technology. 

An image processing apparatus according to an aspect of the 
present invention includes a first converter that subjects an image data 
10 to a first conversion to thereby generate a first image data; a second 
converter that subjects the first image data to a second conversion to 
thereby generate a second image data; an arithmetic unit that conducts 
an arithmetic operation on the image data and the second image data to 
thereby generate a third image data; a compressor that compresses the 
15 third image data to thereby generate a compressed image data; and an 
embedding unit that embeds the compressed image data in the first 
image data. 

An image processing apparatus according to another aspect of 
the present invention includes a converter that subjects an image data 

20 to a predetermined conversion to thereby generate a converted image 
data; an extractor that extracts embedded data, from the image data, 
that is data embedded in the image data; a decompressor that 
decompresses the embedded data extracted to thereby generate a 
compressed embedded data; and an arithmetic unit that conducts an 

25 arithmetic operation on the converted image data the compressed 



embedded data. 

A method of processing image data according to still another 
aspect of the present invention is carried out by an image processing 
apparatus that transmits the image data processed to other apparatus. 
5 The method includes generating an image data; subjecting the image 
data to a first conversion to thereby generate a first image data; 
subjecting the first image data to a second conversion to thereby 
generate a second image data; conducting an arithmetic operation on 
the image data and the second image data to thereby generate a third 

10 image data; compressing the third image data to thereby generate a 

compressed image data; and embedding the compressed image data in 
the first image data. 

A method of processing image data according to still another 
aspect of the present invention is carried out by an image processing 

15 apparatus that receives the image data from other apparatus. The 

method includes subjecting the image data received to a predetermined 
conversion to thereby generate a converted image data; extracting 
embedded data, from the image data, that is data embedded in the 
image data; decompressing the embedded data extracted to thereby 

20 generate a compressed embedded data; conducting an arithmetic 

operation on the converted image data the compressed embedded data 
to thereby generate an arithmetic data; and visualizing the arithmetic 
data. 

Computer-readable recording medium according to still another 
25 aspect of the present invention stores a computer program that makes a 



computer realizes the methods according to the present invention on a 
computer. 

The other objects, features and advantages of the present 
invention are specifically set forth in or will become apparent from the 
following detailed descriptions of the invention when read in conjunction 
with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a general view of an image processing system 
according to an embodiment of the present invention; 

Fig. 2A is a block diagram of a transmission-side image 
processing apparatus according to the embodiment; 

Fig. 2B is a block diagram of a reception-side image processing 
apparatus as a primary target according to the embodiment; 

Fig. 2C is a block diagram of a reception-side image processing 
apparatus, as a secondary target, according to the embodiment; 

Fig. 3A to Fig. 3M are diagrams for explaining specific examples 
of conversions in first, second, and third converters as shown in Fig. 2A 
and Fig. 2C; and 

Fig. 4 is a block diagram of a modification of the reception-side 
image processing apparatus as the secondary target. 

DETAILED DESCRIPTION 

Exemplary embodiments of the present invention are explained 
in detail below with reference to the accompanying drawings. , 



Fig. 1 is a general view of an image processing system 
according to an embodiment of the present invention. 

The image processing system includes a transmission-side 
image processing apparatus (hereinafter, "transmission-side 
5 apparatus") 1 that transmits read-in image data, and a reception-side 
image processing apparatus 5 that receives the image data via a 
communication network 3 and visualizes the image data. The 
reception-side image processing apparatus 5 includes a reception-side 
image processing apparatus (hereinafter, "primary reception^side 
10 apparatus") 5a as a primary target and a reception-side image 
processing apparatus (hereinafter, "secondary reception-side 
apparatus") 5b as a secondary target. 

Fig. 2A is a block diagram of the transmission-side apparatus 1, 
Fig. 2B is a block diagram of the primary reception-side apparatus 5a, 
15 and Fig. 2C is a block diagram of the secondary reception-side 
apparatus 5b. 

As shown in Fig. 2A, the transmission-side apparatus 1 includes 
a data generator 7 that generates digital image data, a first converter 9 
connected to the data generator 7, a second converter 11 connected to 
20 the first converter 9, a subtractor 13 connected to the data generator 7 
and the second converter 11, a compressor 15 connected to the 
subtractor 13, and an embedding unit 17 connected to the first 
converter 9 and the compressor 15. 

The data generator 7 includes a scanner that scans a document 
25 such as a printed matter or a photograph paper, a digital still 



camera/video camera that photographs space, an X-ray camera, a 
magnetic resonance imaging (MRI) device, a computerized tomography 
(CT) scanner, or application software for a raster image processor 
(RIP)/word processor that artificially generates an image. 
5 The first converter 9 and the second converter 11 convert basic 

specification of the image data. Here, the basic specification indicates 
resolution, the number of colors, or a sampling frequency. The first 
converter 9 performs conversion so that values of the specifications are 
reduced (reduction in the resolution, reduction in the number of colors, 

10 and reduction in the sampling frequency). On the other hand, the 
second converter 11 performs conversion so that the values of the 
specifications are increased (increase in the resolution, increase in the 
number of colors, and increase in the sampling frequency). 

The process of reducing the resolution indicates the process of 

15 widening a spatial quantization width, and the process of increasing the 
resolution indicates the process of narrowing the spatial quantization 
width. The process of reducing the number of colors indicates the 
process of widening a color spatial quantization width, and the process 
of increasing the number of colors indicates the process of narrowing 

20 the color spatial quantization width. The process of reducing the 
sampling frequency indicates the process of widening a time 
quantization width, and the process of increasing the sampling 
frequency indicates the process of narrowing the time quantization 
width. 

25 The respective conversions are performed by using known 



image processing methods/ For example, a simple sampling method 
and an averaging method are used for conversion of the resolution to 
lower one. A nearest neighbor method and a load average 
interpolation method are used for conversion of the resolution to higher 
5 one. 

Fig. 3A to Fig. 3M represent specific examples of conversion in 
the first converter 9 and the second converter 11. Fig. 3A represents a 
part of the image data, in which one square represents one pixel and a 
numeral in the square represents a pixel value (gradation), and 4x4 

10 pixels are shown in this example. Fig. 3B represents an image 

converted to a low-resolution image by using the averaging method. 
Fig. 3C and Fig. 3D represent images converted to high-resolution 
images, in which the conversion in Fig. 3C is performed by the nearest 
neighbor method, and the conversion in Fig. 3D is performed by the 

15 load average interpolation method (a load matrix is indicated in Fig. 3M). 
Although the images in Fig. 3C and Fig. 3D are similar to that of Fig. 3A, 
they are not perfectly the same as each other. This indicates the 
example where image data cannot perfectly be restored to the image 
data with original resolution again from the image data whose resolution 

20 has been once reduced. In other words, during the process of 

reducing the resolution, a part of the information has been deleted, and 
therefore, it is impossible to restore the deleted information once again. 

The example of resolution that is divided by 2 and multiplied by 
2 is explained here. However, the present invention is not limited to 

25 this example. For example, when 300 dpi-image data is generated in 



the data generator 7 and the data is transmitted to a 75-dpi monitor as 
the primary target and to a 150-dpi printer as the secondary target, the 
resolution is divided by 4 and is multiplied by 2. Further, the resolution 
is not limited to a second-dimensional resolution, but may be a 
5 three-dimensional resolution. 

The subtractor 13 obtains a difference between two types of 
image data. In the examples of Fig. 3A to Fig. 3M, Fig. subtraction is 
expressed in 3A-Fig. 3C=Fig. 3E and Fig. 3A-Fig. 3D=Fig. 3F. 
The compressor 15 performs entropy encoding after 

10 quantization. Fig. 3G and Fig. 3H are results of quantizing Fig. 3E and 
Fig. 3F, respectively (that is, Fig. 3E and Fig. 3F are divided by 4, and 
each of the remainders is rounded down). The entropy encoding is 
data compression such as arithmetic encoding, Huffman encoding, and 
universal encoding. 

15 The embedding unit 1 7 replaces a redundant part of the image 

data with embedded data using an electronic watermark technology. 
The electronic watermark technology is a process of altering image data 
based on the embedded information within a range in which 
degradation of the image quality is not noticeable. This technology 

20 has been a notable technology in recent years that is applied to 
embedding information such as copy right in a contents image. 

The operation of the transmission-side apparatus 1 is explained 
below with reference to Fig. 2A. 

The image data (Fig. 3A) generated in the data generator 7 is 

25 subjected to lower resolution (Fig. 3B) in the first converter 9, and 



information to be embedded explained later is embedded in the image 
data by the embedding unit 17. Thereafter, the embedded information 
is converted to the standard format by the standard format converter 
(not shown) and the converted information is transmitted to the 
5 reception-side image processing apparatus 5. As the information to be 
embedded, data obtained by compressing differences (Fig. 3E and Fig. 
3F) in the compressor 15 is used. The differences are between data 
whose resolution is reduced in the first converter 9 and increased in the 
second converter 11 (Fig. 3C and Fig. 3D) and the data (Fig. 3A) before 

10 the resolution is reduced. 

Conversion of the embedded information to the standard format 
allows transmitted or received image data to be generalized, which 
makes it possible to ensure use of the data in the reception-side image 
processing apparatus. The standard format includes JPEG (Joint 

15 Photographic Experts Group), JEPG 2000, MPEG (Moving Pictures 
Experts group) 2, and MPEG 4 as the international standard. As the 
information is embedded in the image data, it is not complicated to 
handle a file as compared to the second conventional method in which 
the embedded information is handled as a separate file. 

20 > In the examples of Fig. 3A to Fig. 3M, the entropy, in the case of 

handling 16 quantized values as non-stored information source, is 1.91 
bit/pixel in Fig. 3G and 1.31 bit/pixel in Fig. 3H. Further, if the 
quantized values are handled based on Markov model, the entropy is 
further smaller. If the entropy is 1 bit/pixel, then information of 4 

25 bit/pixel is embedded in the low resolution image of Fig. 3B. If the 



gradation (number of colors) is 24 bit/pixel and the redundancy of the 
image data related to image quality is originally 4/24=16%, this part can 
be replaced with the embedded information without degradation of the 
image quality. 

5 Fig. 2B is a block diagram of the primary reception-side 

apparatus 5a. 

As shown in Fig. 2B, the primary reception-side apparatus 5a as 
the primary target includes a visualizing unit 21 . In other words, as 
image data matching with specifications of equipment as a primary 
10 target is received, the equipment does not particularly need to convert 
the image data, and visualizes the data as it is. The processing load 
of the equipment as a main target is reduced as compared to the first 
conventional method. If information is embedded within the 
redundancy range related to the image quality in the equipment as the 
15 main target, the image quality can be sufficiently ensured in the 
equipment as the main target. 

The visualizing unit 21 visualizes digital image data, and is a 
display unit such as a monitor, projector, and printer. 

Fig. 2C is a block diagram of the secondary reception-side 
20 apparatus 5b. 

The secondary reception-side apparatus 5b includes a third 
converter 23 that receives image data from the transmission-side 
apparatus 1 and converts the basic specification, of the data, and an 
adder 25 connected to the third converter 23. The secondary 
25 reception-side apparatus 5b also includes an extractor 27 that receives 



the image data from the transmission-side apparatus 1 and extracts 
embedded data, a decompressor 29 connected to the extractor 27 and 
the adder 25, and a visualizing unit 21 connected to the adder 25. 

The third converter 23 largely converts resolution as the basic 
5 specifications of the image data transmitted from the transmission-side 
apparatus 1 . 

The extractor 27 extracts the embedded data from the image 
data using the electronic watermark technology. 

The decompressor 29 performs entropy decoding on the image 
10 data and then performs reverse quantization thereon. The images of 
Fig. 3G and Fig. 3H subjected to reverse quantization (x 4) are 
represented by the images of Fig. 31 and Fig. 3J. 

The adder 25 obtains a sum of two types of image data. 
Addition is expressed in Fig. 3C + Fig. 31 = Fig. 3K and Fig. 3D + Fig. 
15 3J = Fig. 3L. 

The visualizing unit 21 visualizes the digital image data, and is a 
display unit such as a monitor, projector, and a printer. 

The operation of the secondary reception-side apparatus 5b (Fig. 
2C) is explained beiow. 
20 The third converter 23 subjects the image data transmitted from 

the transmission-side apparatus 1 through the communication network 
3 to high resolution in order to match with the specifications of the 
secondary reception-side apparatus 5b, and obtains high resolution 
data (Fig. 3C and Fig. 3D). The extractor 27 extracts the embedded 
25 data from the image data transmitted from the transmission-side 



apparatus 1 . The decompressor 29 conducts entropy decoding on the 

embedded data (Fig. 3G and Fig. 3H) and conducts reverse 

quantization thereon to obtain reversely quantized data (Fig. 31 and Fig. 

3J). The adder 25 adds the high resolution data (Fig. 3C and Fig. 3D) 
5 and the reversely quantized data (Fig. 31 and Fig. 3J) to each other to 

obtain data (Fig. 3K and Fig. 3L), and the data is displayed on the 

visualizing unit 21 . 

A difference (average of absolute values of differences between 

pixel values) from the original image (Fig. 3A) is 3.13 in Fig. 3C and 
10 2.88 in Fig. 3D before the addition, whereas the difference becomes 

smaller after the addition such as 0.63 in Fig. 3K and 1.38 in Fig. 3L. 

These figures mean that the addition allows high quality images. 

In other words, a part of the information that has been deleted in 

the transmission-side apparatus to obtain low resolution information is 
15 embedded in the image data in a range in which the embedded 

information does not affect the equipment as the main target. The 

embedded information is extracted and used in the equipment as the 

auxiliary target, and thereby sufficient image quality can be ensured in 

the equipment as the auxiliary target. 
20 Fig. 4 is a block diagram of a modification of the secondary 

reception-side apparatus 5c. 

The secondary reception-side apparatus 5c includes the third 

converter 23 that receives the image data from the transmission-side 

apparatus 1 and converts the basic specification of the data, and the 
25 adder 25 connected to the third converter 23. The reception-side 



apparatus 5c also includes the extractor 27 that receives the image 
data from the transmission-side apparatus 1 and extracts embedded 
information from the image data, the decompressor 29 connected to the 
extractor 27 and the adder 25, and a selector 31 connected to the adder 
5 25. The reception-side apparatus 5c further includes the visualizing 
unit 21 connected to the selector 31, and an embedding determiner 33 
that receives the image data from the transmission-side apparatus 1 to 
determine whether any information is embedded in the image data, and 
transmits the result of determination to the selector 31 . 

10 That is, in the secondary reception-side apparatus 5c, the 

embedding determiner 33 is added to the secondary reception-side 
apparatus 5b of Fig. 2C. The selector 31 selects a process so as to 
perform addition if it is determined by the embedding determiner 33 that 
the information is embedded, and selects another process so as not to 

15 perform the addition if it is determined that the information is not 
embedded. In such a configuration, the secondary reception-side 
apparatus 5c can receive image data also from any device other than 
the transmission-side apparatus 1 of Fig. 2A, which allows the 
reception-side equipment to be more versatile. 

20 The embedding determiner 33 determines whether embedded 

data such as an electronic watermark is included in the image data, and 
generally determines on it depending on whether predetermined pattern 
data can be detected as embedded data. 

Thus, according to the embodiment, the resolution is converted 

25 in the first, second, and third converters 9, 11, and 23, but the number 



of colors and sampling frequency may be converted in the converters 9, 
11, and 23. In this case, the substantially same effect as the examples 
of resolutions can be obtained in terms of the principle of the operation 
although a space to be quantized is different. 
5 The respective units of the transmission-side apparatus 1 and 

the reception-side apparatuses of Fig. 2A to Fig. 2C are illustrated as 
the functional blocks, but the functions may be realized by a computer 
or by reading a computer program stored in a recording medium. 

According to one aspect of the present invention, it is possible 

10 to reduce the processing load, ensure sufficient image quality, ensure 
the use of the image data in the reception-side apparatus, and also 
make the reception-side apparatus more versatile. 

Although the invention has been described with respect to a 
specific embodiment for a complete and clear disclosure, the appended 

15 claims are not to be thus limited but are to be construed as embodying 
all modifications and alternative constructions that may occur to one 
skilled in the art which fairly fall within the basic teaching herein set 
forth. 



16 



